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1.  Introduction 

Consider  a  general  factorial  experiment  with  the  design  con¬ 
sisting  of  t  treatments  and  corresponding  to  the  uth  treatment  there 

t 

are  n  (>  1)  observations  and  Z  n  »  N.  Let  y  be  the  observation 
u  —  ,  u  uv 

u-1 

corresponding  to  the  uth  replication  of  the  vth  treatment  and  be 
the  mean  of  all  observations  corresponding  to  the  uth  treatment.  The 
model  for  this  experiment  is 

E(Z)  -  Bi  +  X2  j$2 , 

V(2)  -  0*1,  (1) 

Rank  XA  ■  vA , 

where  jSj^xl)  is  a  vector  of  specified  lower  order  interactions  and 
$2(v2xl)  is  a  vector  of  some  or  all  of  the  higher  order  interactions, 
X1(Nxv1)  and  X2(Nxv2)  are  known  matrices.  It  is  known  that  K  (very 
small  compared  to  v2)  elements  of  are  nonzero  and  the  other  are 
zero;  however  the  value  of  K  and  the  nonzero  elements  of  are 
unknown.  The  problem  is  to  search  the  nonzero  elements  of  and 
draw  Inferences  on  them  in  addition  to  the  elements  of  j$A .  Such 
a  model  is  called  the  search  linear  model  and  was  introduced  in 


Srlvastava  (1975).  Suppose  Kj  is  an  initial  guess  on  K.  Note  the 
three  possibilities  KA  >  K,  K*  ■  K  and  Kj  <  k.  We  consider  (^2) 
models 


E(£)  -  XA  Bi  +  41)iii>.  i-l,...(jj2), 

V(£)  -  o2I 

RankJx^X^  j  -  v4  +  Kx  , 


(2) 


where  X2  ^(NxKA)  is  a  submatrix  of  X2  and  8?'(Kjxl)  is  a  subvector 

of  $2*  It  can  be  seen  from  Srivastava  (1975)  that  we  in  fact  need 

Bank  [xA,  X^ ,  X^*  ^  ]  ■  (vA  +  2K1),  for  all  i  *  i'.  This  implies 

that  N  (vA  +  2Ka).  In  case  XA  ■  K,  one  of  ('J2)  models  is  the 

M 

correct  model.  If  Kx  >  K,  then  (^2_K)  models  out  of  (K2)  models 
include  the  true  model  as  a  submodel  in  the  expectation  forms  of  the 
models.  The  methods  discussed  in  this  paper  will  not  only  Identify  K 
nonzero  parameters  but  also  find  how  many  of  them  have  significant 
effects  and,  finally,  rank  the  significant  nonegligible  parameters  in 
the  order  of  their  influence  on  the  fitted  values.  In  case  Kj  <  K, 
the  methods  will  identify  from  KA  parameters  the  parameters  which  are 
significant  and  influential.  We  also  propose  an  estimator  of  K  in  the 
Section  3. 

In  some  industrial  experiments,  it  is  often  easy  to  find  replica- 
cations  (n^  _>  1)  in  observations  corresponding  to  a  particular  (the 
uth)  treatments,  see  Taguchi  and  Wu  (1985).  There  are  also  situations 
in  industrial  experiments  where  it  is  impossible  to  get  replication  in 
observations  for  a  treatment,  see  Daniel  (1976)  and  Box  and  Meyer 
(1985).  The  methods  discussed  in  this  paper  consider  both  situations. 
In  all  Taguchi  design  methods,  the  higher  order  interactions  (2-factor 
and  higher  order  in  most  plans)  are  assumed  to  be  zero.  A  few  of 
those  higher  order  Interactions  may  be  nonnegllgible ,  significant  and 
Influential.  The  use  of  the  search  linear  models  may  be  a  potential 
tool  in  Improving  upon  the  Taguchi  design  methods. 
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The  residuals  under  the  model  (1),  when  B?  ■  o,  are 


„(o)  ^(o)  _ 

I  “  ITL  "  pl2* 


It  can  be  seen  that 


Pl  .  z^’z™  ♦  z(1)'z(1>. 


Therefore,  for  i  - 


SSE(°*  -  R(o)V°^  -  SSE^1^  + 


For  1  •  we  define 


X'Z*1*  V^jr/Ki 


SSE,‘1V(N-v1-Ka) 


Let  y^  *  be  the  fitted  value  of  the  observation  corresponding  to  the 


uth  (u  -  l,...,w)  treatment  under  the  1th  model  In  (2).  We  write  the 


sum  of  squares  due  to  lack  of  fit  as 


SSLOF 


r  =  (y-T(1))Z, 

u'  u  u 

U*1 


and  the  sum  of  squares  due  to  pure  error  as 


w  n 

SSPE  -  I  Eu  (y  -y  )  . 

wuv  u 

u-1  v-1 


For  1  -  ^z),  we  define 


F(i)  _  SSLOF(1)/(w-v1-Ki) 
LOF  ”  SSPE/(N-w) 


:  >.  /  ■  **  vh  • 


!WKtv 
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Theorem  1.  For  t  e  { 1 , . . . ,  ( } ,  the  following  statements  are 
- 

equivalent . 

(a)  SSE^*)  is  a  minimum, 

(i) 

(b)  F  is  a  maximum, 

,<*> 


(c)  SSLOF 
<d) 


is  a  minimum, 


(4) 

F^qf  Is  a  minimum. 


(e)  The  Euclidean  distance  between  and  is  a  maximum, 

(f)  The  square  of  the  (sample)  simple  correlation  coefficient  between 

the  elements  of  R^  and  is  a  minimum. 


SSEV 


Proof .  We  have  from  (10)  and  (11)  that 

j<o) 

(N-v.-KV)  ]  SSE(1)  • 

Noting  that  the  numerator  on  the  RHS  of  the  above  expression  does  not 
depend  on  i,  we  get  the  equivalence  of  (a)  and  (b).  Again, 


SSE 


-  SSPE  +  SSL0F(i), 


and  SSPE  does  not  depend  on  i.  Therefore  (a)  and  (c)  are  equivalent. 
From  (14),  the  equivalence  of  (c)  and  (d)  is  clear.  From  (3),  (6), 
(8)  and  (9) ,  it  follow  that 

z-Z<‘>  -  1<1>’x<1)Y1>’z<1>x<1>  i«> 

-!‘1,,x<i),[p1-z<1)'z<1>]  X<‘>!<1> 


g(i)’  (i)»  „(i) g(i) 

22  x2  pix2  £2 


(15) 


-  (R(1)-R<0))’  (R(1>-R(0)) 

-  C-£Ci>^Co>)*  (-£(iV0>). 
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The  equivalence  of  (a)  and  (e)  is  now  easy  to  see  from  (10)  and  (15). 
It  follows  from  (10)  and  (15)  that  R(1)'r(1)  -  R(1)V°*.  We  thus 
have 

SSE(i)  R(i>Vi5  (r(1)Vo))2 

7*7*  '  7*7* '  (£(1>Vl>)  (S‘*,v*>)  °6) 

-  the  square  of  the  (sample)  Simple  correla¬ 
tion  Coefficient  between  R^  and  R^°\ 

The  equivalence  of  (a)  and  (f)  is  now  clear  from  (16).  This  completes 
the  proof  of  the  theorem. 

Propostion  1.  Under  the  1th  model  in  (2), 


Z(1)R(1>  -  o. 


(17) 


Proof .  It  follows  from  (3)  and  (5)  that 


Z<1)z 

This  completes  the  proof. 
We  have 


-  Z«>£<1>. 


V(R(i))  -  o2PA  [l-xi1)(xJ1>,P1^1))  PA. 


(18) 


The  residual  in  R^^  are  correlated  and  the  question  may  be  asked 
about  the  appropriateness  in  combining  the  elements  of  R^  in  SSE^. 
If  we  take  the  transformed  residuals  as  zj^R^,  we  then  have 

E(Z(1i)R(1>)  -  o  and  vfz^V1*)  -  o2!.  (19) 

The  sum  of  squares  of  these  transformed  residuals  is 
(!>•  <i>*  <i>.<l> 


R 


Z|  R 


f 
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Proposltlon  2.  For  i  -  1,...,  » 


SSE 


(i) 


<r(1>,zJ±),zJ1>R^1^  . 


(20) 


Proof.  We  write  the  RHS  using  (9)  as 


mM'iP'iPm™  -  R(1),P1R(1)-  R(1),Z(1)'z(1)R(1), 


(21) 


.(i)  _  M 


It  can  be  checked  that  PjR  ■  R'~' .  By  using  the  Proposition  1,  the 
rest  of  the  proof  is  clear.  This  completes  the  proof. 

?(i> 


Proposition  2  thus  supports  the  use  of  SSE 


Theorem  1  gives 


various  Interpretations  of  a  search  procedure,  discussed  in  Srivastava 

,(*) 


(1975),  of  selecting  *  as  the  influential  set  of  Kx  nonnegliglble 
parameters . 

We  now  denote 


-  r6(i)  b(1)  g(i)i 

I  **21  . P2kJ* 

y(d  rY(i)  y(d  Y(i)i 

X2  |JC21  ,...,X2j  ....  ,X2KJ, 


x|lj) 


,U> 


the  matrix  obtained  from  X2  by 

(i) 


Si2J>  - 


deleting  the  jth  column  of  Xy 


(ij> 


> 


-1 


P(,lj)  -  I  -  x&W’xg”)  xg”’ 


(22) 


p(ij)x(i) 

( i )  12  -  2j 

z  * 


/-(D'.UJ)^!) 


-9- 


It  can  be  seen  that 


Rank 


Z^ 


z(1)' 

-U 


(N  -  V|  -  +  1) , 


(i)’  (!)  v-,  v*,  , 

-U  hi  u  hi  z*  -  ’ 


(23) 


z(1) 

LrU  J 


Mi) 


(i) 


12 


0,  Rank  Z^  -  ^ 


M) 


There  exists  a  nonsingular  (triangular)  matrix  Dq  such  that 

Z(1)  -  dJ1^1*. 

From  (3)  and  (24),  we  have 

54)  -  (z  j1M1>)-1zj1>x. 


(24) 


(25) 


Now 


4i)41>  -  diag  (Zr^  . <2*> 

is  a  diagonal  matrix.  Thus 


«<*>  -  _=Li 


Z(i)’v 
•XU  i 


2J 


Let  R^1^,  i  -  1,...,  (,2).  j  -  1. »Kj ,  be  the  residuals  obtained 


.(!>’  U> 

-lj  -2j 


(27) 


from  ith  model  in  (2)  assuming  8^  *  °*  Then  the  8um  of  squares 


due  to  error  is 


SSE(lj)  -  r(1^),r(1^^  -  (Z^  Z.)2  +  SSE^\  (28) 


We  now  define,  for  i  ■  1#»* •»(-*)  and  j  ■  1,...,KA, 

k  i 
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si 

:h,s 


■>w. 

.?X* 


>5$ 

‘ft; 


1 


1 


spa 


1 

■?;! 

«Vt 


Hi 


.$$ 

iW 


v  5 
:?:!? 


'Vll 


Z(i>’  y 

tUj>  -  -U  1  . 


(N-Vi-Ki) 


Proposition  3.  For  a  fixed  £  in  {l,...,(g2)}  and  an  m  in  {l 


the  following  statements  are  equivalent. 


(a)  SSE^m^  is  a  minimum, 


(b)  t^m^  is  a  maximum. 


Proof .  The  proof  can  be  easily  seen  from  (28)  and  (29). 


(£)  (£) 

In  the  set  0^  °*  influential  nonnegligible  parameters,  0^  is  the 


most  influential  nonnegligible  parameters.  The  Influential  non¬ 


negligible  parameters  may  or  may  not  have  significant  effects  on 


observations. 


f 

LiBiiBmTrmmiTnmMirii  miriini  m  im  i  m  inrun-.  mi  m  ( mrin  ir  r 


3.  Influential  Significant  Nonnegligible  Parameters 


We  now  assume  the  normality  in  (2)  and  therefore  for 

1  -  1 . Q),  y  N(Xlll  +  x«)e<i>, 


Under  the  null  hypothesis  Hq  :  has  the  central  F  distri¬ 

bution  with  (K| ,  N  -  vA  -  Ka)  d.f.  and  under  the  null  hypothesis 
V  has  the  central  t  distribution  with  (N  -  -  Kj) 

d.f.  We  now  present  a  further  development  of  a  procedure  suggested  in 
Srivastava  (1975). 

Case  I.  If  max  <  F„  „  „  ,  we  then  conclude  that  there 

-  ^  —  a;K|  ,n-va-ka 

is  no  significant  nonnegligible  parameter.  (F  is  the 

aJK-l  ,N—  V|—  K.| 

upper  a  percent  point  of  the  central  F  distribution  with 
(Ki ,N-V|-K|)d.f .) . 


Case  II.  Suppose  for  i  *  1^ . ig  ,  we  have  F'  >  Fa>R  n  v  -k 

We  denote  for  j  ■  l,...,v2, 

3,  -  the  number  of  i  in  {i,,...,i  }  for  which  It  I  >  t 
3  18  1  «'  N-v,-*, 

Note  that  0  3^  s.  We  now  arrange  S^'s  in  decreasing  order  of  mag¬ 

nitude  and  write  3/tx  >  3/ON  >  ...  >  3,  N.  If  there  are  at  least  K, 
(1)  -  (2)  —  -  (v2)  1 

nonzero  S^'s,  we  select  the  influential  significant  parameters  as 
0(1),...,S(K  j,  otherwise  we  pick  the  influential  S^'s  corresponding 
to  nonzero  3^’s  (Note  that  the  number  of  Influential  parameters  is 
then  less  than  Kj).  The  parameter  Is  the  most  Influential 

significant  nonnegligible  parameter.  An  estimator  of  the  unknown  K  is 

/s. 

K  ■  the  number  of  nonzero  S^'Sjj  -  l,...,v„. 
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4.  Miscellaneous  Results 


4. a.  Let  us  denote  the  unknown  nonzero  elements  of  *-n  O)  by 

8^  (Kxl)  and  the  zero  elements  of  3^  by  (( v?-K)Xl ) ,  the  cor- 


-2d 

responding  columns  in  matrix  are  and  X^. 
true  expectation  form  of  (1)  is  thus 


The  unknown 


E(y)  -  Xjlj  +  x2ci2c.  (30) 

The  expectation  form  of  the  ith  model  in  (2)  can  be  written  as 
E(Z)  >X16l  +  lt2e>£2c)  +  Jt2d>4d>-  <31) 

,U). 


where  X^^Nxy^)  is  a  submatrix  of  X2c,  x2^  (NxCK^-y^) )  is  a 

submatrix  of  X_,,  8^(Y.xl)  is  a  subvector  of  B0  and  B^V 

((K^-y^Xl)  is  a  subvector  of  J^d*  bet  is  the  vector 

X** 

2< 

(i) 


of  elements  in  B_  which  are  not  in  bI*^  and  X*^^  is  the 
—2c  —2c  2c 


matrix  whose  columns  are  in  X_  but  not  in  X„ 

zc  zc 


The  following 

result,  a  counterpart  of  the  result  in  (10)  for  the  population, 
can  be  verified  very  easily. 

Proposition  4.  Under  (30), 


E(SSE(i))  -  E(SSL0FV1')  +  oZ(N-w) 

-°2<»-vv+^xiczl1),z51)x2di2c 

2  (i)‘  (i)'  (i).  (i)  (i)  (i) 

-  a2(N-vrKl)  +  B$c  X5c  z\  Zx  X|c  B|c 


(i) 


(32) 


E(SSE<0))  -  [oZK1  +  I^cZVi;  ZV1'x2oB2c]  . 


(i)’,(i). 
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4.b.  The  model  obtained  from  (2) 

E(Z(i)Z)  -  Z*1^1^1*, 

v(z(i)y)  -  ®2l» 

is  called  the  pure  search  model  (Srivastava  (1976)).  Xn  fact, 
Srivastava  (1976)  considered  a  special  form  of  Z^\ 

4.c.  The  Influential  nonnegligible  parameter  may  depend  on  noise, 
i.e.,  a  parameter  may  be  influential  under  one  noise  but  may 
not  be  influential  under  another  noise. 


4.d.  The  replicated  observations  will  surely  improve  the  chances  of 
detecting  the  correct  Influential  nonnegligible  parameters. 

4.e.  In  presence  of  outliers  in  observations,  one  may  combine 
residuals  with  unequal  weights,  or  in  other  words,  may  use 
transformed  residuals  (see,  Cook  and  Welsberg  (1982)). 


4.e.l.  An  example  of  transformed  residual^  is  the  vector 

where  M^^(NxN)  is  a  diagonal  matrix  whose  uth 

/  _ _ \  (i) 

diagonal  element  is/  1/  /  (i)  with  m  being  the  uth 

n  v  uu 
_2  (£) 

diagonal  element  of  o  v(r'  ) . 


4.e.2.  Suppose  the  underlying  design  is  robust  against  the 
unavailability  of  any  single  observation  [see,  Ghosh 
(1980)]  in  the  sense  that  the  estimation  of  £x  and  £2^ 


is  possible  under  (2)  when  any  single  observation  is 
unavailable.  We  find  the  predicted  value  of  the  lith 
observation  from  the  remaining  (N-l)  observations  (i.e., 
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by  deleting  the  lith  observation) .  The  difference  between 
the  uth  observation  and  its  predicted  value  is  called  the 
uth  predicted  residual  (using  the  idea  of  cross  valida¬ 
tion).  It  can  be  verified  algebraically  that  the  vector 
of  predicted  residuals  is  [m^]  R^*\  The  predicted 
residual  sun  of  squares  (PRESS)  from  the  1th  model  under 
(2)  is 

PRESS(i)  -  R(i)'[M(i)]  R*1* 

In  presence  of  outliers,  one  may  take  PRESS^  as  an 
alternative  to  SSE^. 
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